Eye-controlled apparatus, eye-controlled method and eye-controlled system

ABSTRACT

The present disclosure discloses an eye-controlled apparatus, an eye-controlled method and an eye-controlled system. The eye-controlled apparatus includes a fixation point acquisition unit, a human eye action detection unit and a control signal generation unit. The fixation point acquisition unit is configured to acquire position information about the fixation point of human eyes on a device to be operated. The human eye action detection unit is configured to detect whether the human eyes take a pre-set action, and control the fixation point acquisition unit to transmit current position information about the fixation point of the human eyes on the device to be operated to the control signal generation unit when detecting the preset action taken by the human eyes. The control signal generation unit is configured to generate, based on a pre-stored position control correspondence table corresponding to the device to be operated, a control signal corresponding to the current position information about the fixation point of the human eyes on the device to be operated, and transmit the control signal to the device to be operated so as to control the device to be operated to execute a corresponding operation. The eye-controlled apparatus, the eye-controlled method and the eye-controlled system may effectively control the device to be operated using human eyes.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Section 371 National Stage Application ofInternational Application No. PCT/CN2016/079259, filed on Apr. 14, 2016,entitled “Eye-Controlled Apparatus, Eye-Controlled Method andEye-Controlled System”, which claims priority to Chinese Application No.201510280294.2, filed on May 27, 2015, incorporated herein by referencein their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a technical filed of display, moreparticularly, relates to an eye-controlled apparatus, an eye-controlledmethod and an eye-controlled system.

Description of the Related Art

Exploring a natural and harmonious human-computer relationship isbecoming more and more important in computer field. A natural, efficientand intelligent human-computer interaction interface is an importanttrend in the development of computer. However, for disabled people orthose who are not idle for their hands (for example, being in activitiessuch as washing their hands, cooking, eating, etc.), it is quitedifficult for them to perform human-computer interaction by using inputdevices such as a mouse, a keyboard, an operator, etc.

In the field of human-computer interaction, since the eye is animportant information exchanging channel and the sight line of the humanmay reflect attention direction of the human, the sight line applied tothe field of human-computer interaction obtains the natural, direct andinteractive features, thereby gaining many People's attention. How torealize human-computer interaction based on human eye action is animportant research item in this field.

SUMMARY OF THE INVENTION

The present invention provides an eye-controlled apparatus, aneye-controlled method and an eye-controlled system, which may controlthe devices to be operated based on the action of the human eyes.

In order to realize the above-mentioned purpose, the present disclosureprovides an eye-controlled apparatus, comprising:

a fixation point acquisition unit configured to acquire positioninformation about the fixation point of human eyes on a device to beoperated;

a human eye action detection unit configured to detect whether the humaneyes take a preset action, and control the fixation point acquisitionunit to output current position information about the fixation point ofthe human eyes on the device to be operated when detecting the presetaction taken by the human eyes;

a control signal generation unit configured to generate, based on apre-stored position control correspondence table corresponding to thedevice to be operated, a control signal corresponding to the currentposition information about the fixation point of the human eyes on thedevice to be operated, and transmit the control signal to the device tobe operated so as to control the device to execute a correspondingoperation,

wherein position information on the device to be operated and thecontrol signals each corresponding to each of position information arestored in the position control correspondence table.

Optionally, the fixation point acquisition unit comprises:

an infrared emission module provided on the device to be operated andadapted to emit infrared light to the human eyes and form lightreflection points in two pupils of the human eyes;

a first human eye image acquisition module provided on the device to beoperated and adapted to acquire an image of the human eyes;

an image processing module configured to establish an image coordinatesystem based on the image of the human eyes acquired by the first humaneye image acquisition module, and process the image of the human eyes toobtain position coordinates of centers of the two pupils and of thelight reflection points in the image coordinate system; and

a calculation module configured to obtain the position information aboutthe fixation point of the human eyes on the device to be operated by across ratio invariance algorithm based on the position coordinates ofcenters of the two pupils and of the light reflection points in theimage coordinate system obtained by the image processing module.

Optionally, the infrared emission module comprises four infraredemission sources provided at four corners of the device to be operatedrespectively.

Optionally, the fixation point acquisition unit further comprises acorrection module configured to correct the position information aboutthe fixation point of the human eyes on the device to be operatedaccording to a linear scaling algorithm.

Optionally, the image processing module and the calculation module areintegrated in the device to be operated.

Optionally, the fixation point acquisition unit comprises:

spectacles;

a scene acquisition module provided on the spectacles and adapted toacquire a scene image seen by the human eyes through the spectacles, thescene image containing an image of the device to be operated;

a second human eye image acquisition module provided on the spectaclesand adapted to acquire an image of the human eyes,

a fixation direction determination module configured to determine afixation direction of the human eyes based on the image of the humaneyes acquired by the second human eye image acquisition module; and

a fixation point determination module configured to determine theposition information about the fixation point of the human eyes on thedevice to be operated based on the scene image acquired by the sceneacquisition module and the fixation direction of the human eyesdetermined by the fixation direction determination module.

Optionally, the fixation direction determination module is configured todetermine the fixation direction of the human eyes based on pupilpositions in the image of the human eyes acquired by the second humaneye image acquisition module.

Optionally, the preset action comprises keeping the position of thefixation point of the human eyes on the device to be operated unchangedfor 2 to 3 seconds, or, blinking rapidly for 3 to 5 times within apreset time while keeping the position of the fixation point of thehuman eyes on the device to be operated unchanged.

Optionally, the human eye action detection unit is an apparatus havingan imaging function.

Optionally, the human eye action detection unit is configured to acquirea real-time image of the human eyes by means of the first human eyeimage acquisition module or the second human eye image acquisitionmodule, and obtain the action of the human eyes based on the real-timeimage of the human eyes.

In order to realize the above-mentioned purpose, the present disclosurealso provides an eye-controlled system comprising a plurality of devicesto be operated and an eye-controlled apparatus as mentioned above.

Optionally, when the fixation point acquisition unit in theeye-controlled apparatus comprises: the infrared emission module, thefirst human eye image acquisition module, the image processing moduleand the calculation module as mentioned above, the fixation pointacquisition units in the eye-controlled apparatus have the same numberas that of the devices to be operated, and are one-to-one correspondingto the devices to be operated; and each fixation point acquisition unitsis provided on a respective device to be operated.

Optionally, the human eye action detection units in the eye-controlledapparatus have the same number as that of the devices to be operated,and are one-to-one corresponding to the devices to be operated; and eachhuman eye action detection units is provided on a respective device tobe operated.

Optionally, all of the fixation point acquisition units are connected toone single control signal generation unit, which is adapted to store aplurality of position control correspondence tables corresponding to thedevices to be operated, respectively, and transmit control informationto each device to be operated.

Optionally, when the fixation point acquisition unit in theeye-controlled apparatus comprises: the spectacles, the sceneacquisition module, the second human eye image acquisition module, thefixation direction determination module and the fixation pointdetermination module as mentioned above, the eye-controlled apparatuscomprises one fixation point acquisition unit; and

the eye-controlled system further comprises a plurality of activationunits one-to-one corresponding to the devices to be operated; eachactivation units is provided on a respective device to be operated,configured to activate the eye-controlled apparatus when the fixationpoint acquisition unit is directed to a device to be operatedcorresponding to the activation unit, and control the control signalgeneration unit to call the position control correspondence tablecorresponding to the device to be operated corresponding to theactivation unit.

Optionally, the eye-controlled apparatus comprises one human eye actiondetection unit provided on the spectacles of the fixation pointacquisition unit.

In order to realize the above-mentioned purpose, the present disclosurealso provides an eye-controlled method performed based on aneye-controlled apparatus as mentioned above, the eye-controlled methodcomprising steps of:

by the fixation point acquisition unit, acquiring the positioninformation about the fixation point of the human eyes on the device tobe operated;

by the human eye action detection unit, detecting whether the human eyestake the preset action, and controlling the fixation point acquisitionunit to transmit the current position information about the fixationpoint of the human eyes on the device to be operated to the controlsignal generation unit when detecting the preset action taken by thehuman eyes; and

by the control signal generation unit, generating the control signalcorresponding to the current position information about the fixationpoint of the human eyes on the device to be operated based on thepre-stored position control correspondence table corresponding to thedevice to be operated, and transmitting the generated control signal tothe device to be operated to control the device to be operated toexecute a corresponding operation,

wherein the position information on the device to be operated and thecontrol signals corresponding to the position information, respectively,are stored in the position control correspondence table.

Optionally, the eye-controlled method further comprises a step of: priorto acquiring the position information about the fixation point of thehuman eyes on the device to be operated by the fixation pointacquisition unit, establishing the position control correspondence tablecorresponding to the device to be operated.

The technical solution disclosed in the present disclosure has thefollowing advantages:

The present disclosure provides an eye-controlled apparatus, aneye-controlled method and an eye-controlled system, wherein theeye-controlled apparatus comprises a fixation point acquisition unit, ahuman eye action detection unit and a control signal generation unit.The fixation point acquisition unit is configured to acquire positioninformation about the fixation point of the human eyes on a device to beoperated; the human eye action detection unit is configured to detectwhether the human eyes take a pre-set action, and control the fixationpoint acquisition unit to transmit current position information aboutthe fixation point of the human eyes on the device to be operated to thecontrol signal generation unit when detecting the preset action taken bythe human eyes; and the control signal generation unit is configured togenerate a control signal corresponding to the current positioninformation about the fixation point of the human eyes on the device tobe operated based on a pre-stored position control correspondence tablecorresponding to the device to be operated, and transmit the controlsignal to the device to be operated so as to control the device to beoperated to execute a corresponding operation. The technical solution ofthe present invention may effectively utilize the human eyes to controla device to be operated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of an eye-controlled apparatusaccording to an embodiment of the present invention;

FIG. 2 is a schematic view of a particular application of theeye-controlled apparatus according to an embodiment of the presentinvention;

FIG. 3 is a schematic view of a fixation point acquisition unit in aneye-controlled apparatus according to an embodiment of the presentinvention;

FIG. 4 is a further schematic view of a fixation point acquisition unitin an eye-controlled apparatus according to an embodiment of the presentinvention;

FIG. 5 is a schematic view of an eye-controlled system according to anembodiment of the present invention;

FIG. 6 is a further schematic view of an eye-controlled system accordingto an embodiment of the present invention; and

FIG. 7 is a flow chart of an eye-controlled method according to anembodiment of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

The eye-controlled apparatus, the eye-controlled system and theeye-controlled method provided by the present invention will bedescribed in detail hereinafter with reference to the accompanyingdrawings in order to provide a better understanding of the technicalsolution of the present invention by those skilled in the art.

FIG. 1 is a schematic block diagram of an eye-controlled apparatusaccording to an embodiment of the present invention. As shown in FIG. 1,the eye-controlled apparatus comprises: a fixation point acquisitionunit 1, a human eye action detection unit 2 and a control signalgeneration unit 3. The fixation point acquisition unit 1 is connected tothe human eye action detection unit 2 and the control signal generationunit 3. The fixation point acquisition unit 1 is configured to acquireposition information about the fixation point of human eyes on a deviceto be operated; and the human eye action detection unit 2 is configuredto detect whether the human eyes take a pre-set action, and control thefixation point acquisition unit 1 to transmit current positioninformation about the fixation point of the human eyes on the device tobe operated to the control signal generation unit 3 when detecting thepreset action taken by the human eyes; and the control signal generationunit 3 is configured to generate, based on a pre-stored position controlcorrespondence table corresponding to the device to be operated, acontrol signal corresponding to the current position information aboutthe fixation point of the human eyes on the device to be operated, andtransmit the generated control signal to the device to be operated so asto control the device to be operated to execute a correspondingoperation. It should be noted that the position information on thedevice to be operated and the control signals corresponding to positioninformation, respectively, are stored in the position controlcorrespondence table.

In the eye-controlled apparatus according to the embodiment, firstly,the position information about the fixation point of the human eyes onthe device to be operated is acquired by the fixation point acquisitionunit 1. Next, the human eye action detection unit 2 detects anddetermines whether the human eyes take a pre-set action, and controlsthe fixation point acquisition unit 1 to transmit the current positioninformation about the fixation point of the human eyes on the device tobe operated to the control signal generation unit 3 when detecting thepreset action taken by the human eyes. Finally, the control signalgeneration unit 3 generates, based on the pre-stored position controlcorrespondence table corresponding to the device to be operated, thecontrol signal corresponding to the current position information aboutthe fixation point of the human eyes on the device to be operated, andcontrol the device to be operated to execute the corresponding operationbased on the control signal, so as to implement the eye control.

It should be noted that the device to be operated in various embodimentsof the present invention may comprise an electronic device such as amobile phone, a touch display screen, a PAD (tablet), or the like. The“connection” in various embodiments of the present invention maycomprise either a wired electrical connection or a wireless electricalconnection, and is not limited thereto.

The specific operation for realizing eye control by the eye-controlledapparatus provided by the embodiment of the present invention will bedescribed in detail with reference to some examples. It is assumed thatthe device to be operated is a touch display screen in which a pluralityof soft keys with fixed positions are provided in the display area ofthe touch display screen and a plurality of physical keys with fixedposition are provided in the peripheral area.

FIG. 2 is a schematic view of a particular application of theeye-controlled apparatus according to an embodiment of the presentinvention. It is easy to understand that a position controlcorrespondence table corresponding to the touch display screen 4 shouldbe established before the eye control operation is performed by theeye-controlled apparatus provided by the embodiment of the presentinvention. It should be noted that in the present embodiment, theposition information in the position control correspondence table isspecifically described as coordinate information by way of example.However, it should be understood by those skilled in the art that it isonly exemplary embodiment to describe the position information in thepresent embodiment as the coordinate information, which does not limitthe technical aspects of the embodiments of the present invention. In anembodiment of the present invention, any information that can be used todescribe the specific position of the fixation point of the human eyeson the device to be operated may be used as the position informationabout the fixation point of the human eyes on the device to be operated.

In the present embodiment, when the position control correspondencetable corresponding to the touch display screen 4 is established, thecoordinate system may be established in such a way that a lower leftcorner of the touch display screen 4 is described as an origin point, adirection extending from the lower left corner of the touch displayscreen 4 to the lower right corner thereof is described as an X-axis,and a direction extending from the lower left corner of the touchdisplay screen 4 to the upper left corner thereof is described as a Yaxis. Then, the entire touch display screen 4 is coordinated so that nsampling points uniformly distributed and located at different positionsmay be provided on the entire touch display screen 4. Specifically, nsampling points at different positions may be provided on the touchdisplay screen and constitute a sampling point array with a rows and bcolumns. The distances between any two sampling points adjacent to eachother in the row direction or in the column direction in this samplingpoint array are equal to each other. It should be noted that the nsampling points should cover all the keys (soft keys and physical keys)on the touch display screen 4, that's to say, the area of each key (softand physical keys) on the touch display screen 4 should include at leastone sampling point. After setting the sampling points, each controlsignal is set to correspond to the position of each sampling point so asto generate the position control correspondence table corresponding tothe touch display screen 4, and the position control correspondencetable is stored in the control signal generation unit 3. An example ofthe generated position control correspondence table corresponding to thetouch display screen 4 is shown in Table 1 below.

TABLE 1 Position information Control signal (X₁, Y₁) Control signal forno operation (X₂, Y₂) Control signal for Volume Up (X₃, Y₃) Controlsignal for Volume Up (X₄, Y₄) Control signal for Volume Down (X₅, Y₅)Control signal for Volume Down (X₆, Y₆) Control signal for ON/OFF . . .. . . . . . . . . (X_(n−2), Y_(n−2)) Control signal for inputtingcharacter A (X_(n−1), Y_(n−1)) Control signal for no operation (X_(n),Y_(n)) Control signal for no operation

Specifically, when the touch display screen 4 receives the controlsignal for no operation (i.e., on the touch display screen 4 is notprovided with a corresponding key (soft key and physical key) at thecurrent position of the fixation point of the human eyes), the touchdisplay screen 4 does not perform any operation; when the touch displayscreen 4 receives the control signal for Volume Up, the touch displayscreen 4 will perform an operation of rising up the volume; and when thetouch display screen 4 receives the control signal for inputtingcharacter “A”, the touch display screen 4 will perform an operation ofinputting the character “A” at a preset position on the display screen,and so on. They will not be illustrated one by one.

As can be seen from Table 1 above, one position information correspondsto only one control signal, whereas one control signal may correspond totwo or more different position information. For example, the coordinate(X₂, Y₂) and the coordinate (X₃, Y₃) all correspond to the controlsignal for Volume Up, since some keys on the touch display screen 4 maybe larger, and when there is a large number of sampling points, the keywith larger size may cover two or more sampling points. In this case,when the fixation point of the human eyes on the touch display screen 4falls at any of the two or more sampling points, it is determined thatthe user expects to make an operating on the key with larger size. Forexample, when the fixation point of the human eyes on the touch displayscreen 4 falls at least one of the sampling points with coordinate (X₂,Y₂) and (X₃, Y₃), it is possible to determine that the user expects tooperate the Volume Up key.

The operation of the eye-controlled apparatus provided in the presentembodiment will be described in detail with reference to FIG. 2 bytaking that the eye-controlled apparatus is to control the touch displayscreen 4 to rise up the Volume according to the action of the human eyesas an example.

First, the user moves the fixation point on the touch display screen 4to the area of the Volume Up key on the touch display screen 4. At thistime, the fixation point acquisition unit 1 acquires the positioninformation about the fixation point of the human eyes on the touchdisplay Screen 4. For example, the coordinate of the fixation point ofthe human eyes on the touch display Screen 4 acquired by the fixationpoint acquisition unit 1 is coordinate (X₂, Y₂) or (X₃, Y₃). In thepresent embodiment, the coordinate obtained by the fixation pointacquisition unit 1 being coordinate (X₂, Y₂) will be described as anexample.

FIG. 3 is a schematic view of the fixation point acquisition unit in theeye-controlled apparatus according to an embodiment of the presentinvention. As shown in FIG. 3, as an selectable structure of thefixation point acquisition unit in the eye-controlled apparatusaccording to the present embodiment (the structure is used in a casewhere the user does not wear glasses, i.e., the case of the naked eye),the fixation point acquisition unit 1 comprises: an infrared emissionmodule 6, a first human eye image acquisition module 7, an imageprocessing module 8 and a calculation module 9. The infrared emissionmodule 6 is provided on the device to be operated 5, such as, the touchdisplay Screen 4, and adapted to emit infrared light to the human eyesand form light reflection points in two pupils of the human eyes.Specifically, the infrared emission module 6 comprises four infraredemission sources uniformly distributed at four corners of the device tobe operated 5. The first human eye image acquisition module 7 is alsoprovided on the device to be operated 5 and adapted to acquire an imageof human eyes (the human eye image). The image processing module 8 iselectrically connected to the first human eye image acquisition module 7and is configured to establish an image coordinate system based on theimage of the human eyes acquired by the first human eye imageacquisition module 7, and process the image of the human eyes to obtainposition coordinates of centers of the two pupils and of the lightreflection points in the image coordinate system. The calculation module9 is electrically connected to the image processing module 8, and isconfigured to obtain the position information about the fixation pointof the human eyes on the device to be operated 5 by a cross ratioinvariance algorithm based on the position coordinates of centers of thetwo pupils and of the light reflection points in the image coordinatesystem obtained by the image processing module 8. Further, the fixationpoint acquisition unit 1 of the present embodiment comprises acorrection module configured to correct the position information aboutthe fixation point of the human eyes on the device to be operated 5according to a linear scaling algorithm, so as to obtain a preciseposition of the fixation point of the human eyes on the device to beoperated 5.

It should be further noted that the image processing module 8 and thecalculation module 9 in the fixation point acquisition unit 1 in FIG. 3may be integrated in a respective device to be operated 5.

FIG. 4 is a further schematic view of the fixation point acquisitionunit in the eye-controlled apparatus according to an embodiment of thepresent invention. As shown in FIG. 4, as a further selectable structureof the fixation point acquisition unit in the eye-controlled apparatusaccording to the present embodiment (the structure is used in a casewhere the user wears spectacles), the fixation point acquisition unit 1comprises: spectacles 10; a scene acquisition module 11, a second humaneye image acquisition module12, a fixation direction determinationmodule 13 and a fixation point determination module 14. The sceneacquisition module 11 is provided on the spectacles 10 and adapted toacquire a scene image seen by the human eyes through the spectacles 10,wherein the scene image contains an image of the device to be operated 5(such as, the touch display Screen 4). The second human eye imageacquisition module 12 is provided on the spectacles 10 and adapted toacquire an image of the human eyes. The fixation direction determinationmodule 13 is electrically connected to the second human eye imageacquisition module 12, and is configured to determine a fixationdirection of the human eyes based on the image of the human eyesacquired by the second human eye image acquisition module 12. Thefixation point determination module 14 is electrically connected to thescene acquisition module 11 and the fixation direction determinationmodule 13, and is configured to determine the position information aboutthe fixation point of the human eyes on the device to be operated 5based on the scene image acquired by the scene acquisition module 11 andthe fixation direction of the human eyes determined by the fixationdirection determination module 13.

Optionally, the fixation direction determination module 13 is furtherconfigured to determine the fixation direction of the human eyes basedon pupil positions in the image of the human eyes acquired by the secondhuman eye image acquisition module 12.

It should be further noted that the above-mentioned two structures ofthe fixation point acquisition unit 1 provided by the embodiments of thepresent invention are only exemplary structure, and the fixation pointacquisition unit 1 in a further embodiment of the present invention mayalso use other devices that may acquire the position information aboutthe fixation point of the human eyes on the device to be operated, suchas a naked eye tracker or a wearable eye tracker.

After moving the position of the fixation point on the touch display 4to the area of the Volume Up key on the touch display screen 4, the usercontrols the eye to take a preset action, for example, keeping theposition of the fixation point of the human eyes on the touch displayscreen 4 unchanged for 2 to 3 seconds, or, blinking rapidly for 3 to 5times within a preset time while keeping the position of the fixationpoint of the human eyes on the touch display screen 4 unchanged. Itshould be noted that the preset action in the present embodiment is notlimited to the above-described two examples, but may be set inaccordance with the needs of the user.

When the human eye action detection unit 2 detects the preset actiontaken by the human eyes, the human eye action detection unit 2 controlsthe fixation point acquisition unit 1 to transmit current positioninformation about the fixation point of the human eyes on the device tobe operated 5 to the control signal generation unit 3. Specifically, thehuman eye action detection unit 2 controls the fixation pointacquisition unit 1 to transmit the current position coordinate (X₂, Y₂)of the fixation point of the human eyes on the device to be operated 5to the control signal generation unit 3.

It should be noted that the human eye action detection unit 2 in thepresent embodiment may comprise an apparatus having an imaging functionsuch as a CCD (Charge-Coupled Device). Optionally, the human eye actiondetection unit 2 in the present embodiment may acquire a real-time imageof the human eyes by means of the first human eye image acquisitionmodule 7 or the second human eye image acquisition module 12, and obtainthe action of the human eyes based on the real-time image of the humaneyes.

After receiving the current position information about the fixationpoint of the human eyes on the device to be operated 5, the controlsignal generation unit 3 generates the control signal corresponding tothe current position information based on the pre-stored positioncontrol correspondence table corresponding to the device to be operated5. Specifically, the control signal generation unit 3 generates thecontrol signal for Volume Up corresponding to the current positioncoordinate (X₂, Y₂), based on the position control correspondence table(for example, as shown in FIG. 1) corresponding to the touch displayscreen 4 and the current position coordinate (X₂, Y₂), and thentransmits the control signal for Volume Up to the touch display screen4.

After receiving the control signal for Volume Up, the touch displayscreen 4 automatically performs the operation for Volume Up to raise thevolume and finishes the eye control process.

The connection between the human eye sight line and the device to beoperated 5 may be effectively established by the eye-controlledapparatus provided by the present embodiment, thereby realizing thecontrol to the device to be operated 5 by the human eyes and improvingthe experience of human-computer interaction.

The present invention further provides an eye-controlled systemcomprising an eye-controlled apparatus and a plurality of devices to beoperated, wherein the eye-controlled apparatus employs an eye-controlledapparatus described in any one of the above-described embodiments, andthe detailed description thereof is made referring to theabove-mentioned embodiments, and will not be described again herein.

FIG. 5 is a schematic view of an eye-controlled system according to anembodiment of the present invention. As shown in FIG. 5, as analternative embodiment of the eye-controlled system provided by theembodiment of the present invention, the fixation point acquisition unit1 in the eye-controlled apparatus of the eye-controlled system uses thefixation point acquisition unit 1 shown in FIG. 3, so as to realize anaked eye control, and the specific structure of the fixation pointacquisition unit 1 may be referred to the description of thecorresponding contents in the above embodiment, and will not bedescribed again herein. The eye-controlled system as shown in FIG. 5comprises a plurality of devices to be operated (e.g., the device to beoperated A, the device to be operated B, . . . , the device to beoperated Z), the devices to be operated in the eye-controlled apparatushave the same number as that of the fixation point acquisition units 1,and are one-to-one corresponding to the fixation point acquisition units1; and each fixation point acquisition unit 1 is provided on arespective device to be operated so as to acquire the positioninformation about the fixation point of the human eyes on the respectivedevice to be operated 5 when the user is performing the eye control.

In the present embodiment, all of the fixation point acquisition units 1are electrically connected to a same control signal generation unit 3,and the control signal generation unit 3 is configured to store aplurality of position control correspondence tables corresponding to thedevices to be operated, respectively, (that is, the position controlcorrespondence tables have a same number as that of the number of thedevices to be operated, and are one-to-one corresponding to the devicesto be operated), and to transmit the control information to each devicesto be operated.

Optionally, in the present embodiment, the human eye action detectionunits 2 in the eye-controlled apparatus have the same number as that ofthe devices to be operated, and are one-to-one corresponded to thedevices to be operated; and each human eye action detection units 2 isprovided on a respective device to be operated so as to acquire theimage of the human eyes in front of the respective device to be operatedand detect whether the user makes the preset action based on the imageof the human eyes when the user is performing the eye control.

The operation of the eye-controlled system provided by the embodiment ofthe present invention, taking that the user is to control the device Bto be operated as shown in FIG. 5 in the naked eye control way as anexample, will be described in detail hereinafter.

When operating the device B, the user moves to the position just facingthe device B and move the position of the fixation point of the eye onthe device B to an area of a certain key (soft key and physical key) tobe operated on the device B, and then control the eyes to take thepreset action. At this time, the fixation point acquisition unit 1provided on the device B may acquire the position information about thefixation point of the human eyes on the device B (the fixation pointacquisition unit 1 on the other devices cannot detect the positioninformation about the fixation point of the human eyes thereon). Thehuman eye action detection unit 2 corresponding to the device B maydetect that the human eyes have taken a preset action with respect tothe device B, and the human eye action detection unit 2 corresponding tothe device B controls the fixation point acquisition unit 1 on thedevice B to transmit the current position information about the fixationpoint of the human eyes on the device t B to the control signalgeneration unit 3. After receiving the current position informationtransmitted from the fixation point acquisition unit 1 on the device B,the control signal generation unit 3 calls the position controlcorrespondence table corresponding to the device B, generates thecontrol signal corresponding to the current position information andadapted to control the device B, based on the received current positioninformation and the called position control correspondence tablecorresponding to the device B, and transmits the control signal to thedevice B so as to control the device B to execute a correspondingoperation.

It should be noted that in the eye-controlled system of FIG. 5, thehuman eye action detection unit 2 may acquire a real-time image of thehuman eyes by means of the first human eye image acquisition module 7,and obtain the action of the human eyes based on the real-time image ofthe human eyes, and then detect whether the human eyes have taken thepreset action with respect to the device corresponding to the human eyeaction detection unit 2.

FIG. 6 is a further schematic view of an eye-controlled system accordingto an embodiment of the present invention. As shown in FIG. 6, as afurther alternative embodiment of the eye-controlled system provided bythe embodiment of the present invention, the fixation point acquisitionunit 1 in the eye-controlled apparatus of the eye-controlled system mayuse the fixation point acquisition unit 1 shown in FIG. 4, so as torealize a wearable eye control, and the specific structure of thefixation point acquisition unit 1 may be referred to the description ofthe corresponding contents in the above embodiment, and will not bedescribed again herein. The eye-controlled system as shown in FIG. 6comprises a plurality of devices to be operated (e.g., the device to beoperated A, the device to be operated B . . . the device to be operatedZ), and the eye-controlled apparatus comprises only one the fixationpoint acquisition unit 1. The eye-controlled system further comprises: aplurality of activation units 15 having the same number as that of thedevices to be operated and one-to-one corresponding to the devices to beoperated, and each activation units 15 is provided on a respectivedevice to be operated. Each activation units 15 is configured toactivate the eye-controlled apparatus when the fixation pointacquisition unit 1 is directed to a device to be operated correspondingto the activation unit 15, and control the control signal generationunit 3 to call a position control correspondence table corresponding tothe device to be operated corresponding to the activation unit 15.

Optionally, in the present embodiment, the eye-controlled apparatuscomprises one human eye action detection unit 2 provided on thespectacles 10 of the fixation point acquisition unit 1.

The operation of the eye-controlled system provided by the embodiment ofthe present invention, in which the user will control the device B to beoperated as shown in FIG. 6 in the wearable eye control way, will bedescribed in detail hereinafter.

When operating the device B, the user wears the spectacle type fixationpoint acquisition unit 1 (at this time, the whole eye-controlledapparatus are in non-work status), and move to the position just facingthe device and move the position of the fixation point of the eye on thedevice B to the area of a certain key (soft key and physical key) to beoperated on the device B, and then control the eyes to take the presetaction. At this time, the activation unit 15 provided on the device Bdetects that the fixation point acquisition unit 1 is just facing thedevice, and then activates the eye-controlled apparatus, that is, theeye-controlled apparatus is started to work, and controls the controlsignal generation unit 3 to call a position control correspondence tablecorresponding to the device. Meanwhile, since the fixation pointacquisition unit 1 and the human eye action detection unit 2 in theeye-controlled apparatus begin to work, the fixation point acquisitionunit 1 may acquire the position information about the fixation point ofthe human eyes on the device B, and the human eye action detection unit2 may detect that the human eyes have taken a preset action, and controlthe fixation point acquisition unit 1 to transmit the current positioninformation about the fixation point of the human eyes on the device Bto the control signal generation unit 3. After receiving the currentposition information about the fixation point of the human eyes on thedevice B transmitted from the fixation point acquisition unit 1, thecontrol signal generation unit 3 generates the control signalcorresponding to the received current position information, based on thereceived current position information and the previously called positioncontrol correspondence table corresponding to the device B, and thentransmits the control signal to the device so as to control the device Bto execute a corresponding operation.

It should be noted that in the eye-controlled system as shown in FIG. 6,the human eye action detection unit 2 may be wirelessly connected to thesecond human eye image acquisition module 12, and acquire the real-timeimage of the human eyes by means of the second human eye imageacquisition module 12, and obtain the action of the human eyes based onthe real-time image of the human eyes, and then detect whether the humaneyes have taken the preset action with respect to the facing device tobe operated. Certainly, the control signal generation unit 3 in thepresent embodiment may also be wirelessly connected to the fixationpoint determination module 14 in the fixation point acquisition unit 1(see FIG. 4) so as to receive the position information transmitted fromthe fixation point determination module 14.

The eye-controlled system provided by the embodiment of the presentinvention has a simple structure and may control the plurality ofdevices to be operated by using only one control signal generation unit,such that the cost of the entire eye-controlled system may be greatlyreduced.

FIG. 7 is a flow chart of an eye-controlled method according to anembodiment of the present invention. As shown in FIG. 7, theeye-controlled method is based on the eye-controlled apparatus providedby the above-mentioned embodiments, and the specific structure of theeye-controlled apparatus may be referred to the description in the aboveembodiment, and will not be described again herein. The eye-controlledmethod comprises the following steps:

Step 101: by the fixation point acquisition unit, acquiring the positioninformation about the fixation point of the human eyes on the device tobe operated;

Step 102: by the human eye action detection unit, detecting whether thehuman eyes take the preset action, and controlling the fixation pointacquisition unit to transmit the current position information about thefixation point of the human eyes on the device to be operated to thecontrol signal generation unit when detecting the preset action taken bythe human eyes;

Step 103: by the control signal generation unit, generating the controlsignal corresponding to the current position information about thefixation point of the human eyes on the device to be operated based onthe pre-stored position control correspondence table corresponding tothe device to be operated, and transmitting the generated control signalto the device to be operated so as to control the device to execute acorresponding operation.

It should be noted that the position information on the device to beoperated and the control signals corresponding to the positioninformation, respectively, are stored in the position controlcorrespondence table.

In an embodiment, before step 101, the eye-controlled method furthercomprises the step of establishing the position control correspondencetable corresponding to the device to be operated.

In the eye-controlled method of the embodiments of the presentinvention, firstly the fixation point acquisition unit acquires theposition information about the fixation point of the human eyes on thedevice to be operated; next, the human eye action detection unit detectswhether the human eyes perform the preset action, and control thefixation point acquisition unit to transmit the current positioninformation about the fixation point of the human eyes on the device tobe operated to the control signal generation unit when detecting thepreset action taken by the human eyes; and finally the control signalgeneration unit generates the control signal corresponding to thecurrent position information about the fixation point of the human eyeson the device to be operated based on the position controlcorrespondence table corresponding to the device to be operated, andtransmits the generated control signal to the device to be operated soas to control the device to execute a corresponding operation, such thatthe human eyes may control the device.

It should be understood that the above embodiments are merelyillustrative embodiments for the purpose of illustrating the principlesof the invention, but the invention is not limited thereto. It will beapparent to those skilled in the art that various changes andmodifications can be made therein without departing from the spirit andspirit of the invention, which are also considered to be within thescope of the present invention.

1. An eye-controlled apparatus, comprising: A fixation point acquisitionunit configured to acquire position information about a fixation pointof human eyes on a device to be operated; a human eye action detectionunit configured to detect whether the human eyes take a pre-set action,and control the fixation point acquisition unit to output currentposition information about the fixation point of the human eyes on thedevice to be operated when detecting the preset action taken by thehuman eyes; and a control signal generation unit configured to generate,based on a pre-stored position control correspondence tablecorresponding to the device to be operated, a control signalcorresponding to the current position information about the fixationpoint of the human eyes on the device to be operated, and transmit thecontrol signal to the device to be operated so as to control the deviceto execute a corresponding operation, wherein position information onthe device to be operated and the control signals corresponding toposition information are stored in the position control correspondencetable, respectively.
 2. The eye-controlled apparatus according to claim1, wherein the fixation point acquisition unit comprises: an infraredemission module provided on the device to be operated and adapted toemit infrared light to the human eyes and form light reflection pointsin two pupils of the human eyes; a first human eye image acquisitionmodule provided on the device to be operated and adapted to acquire animage of the human eyes; an image processing module configured toestablish an image coordinate system based on the image of the humaneyes acquired by the first human eye image acquisition module, andprocess the image of the human eyes to obtain position coordinates ofcenters of the two pupils and of the light reflection points in theimage coordinate system; and a calculation module configured to obtainthe position information about the fixation point of the human eyes onthe device to be operated by a cross ratio invariance algorithm based onthe position coordinates of centers of the two pupils and of the lightreflection points in the image coordinate system obtained by the imageprocessing module.
 3. The eye-controlled apparatus according to claim 2,wherein the infrared emission module comprises four infrared emissionsources provided at four corners of the device to be operated,respectively.
 4. The eye-controlled apparatus according to claim 2,wherein the fixation point acquisition unit further comprises acorrection module configured to correct the position information aboutthe fixation point of the human eyes on the device to be operatedaccording to a linear scaling algorithm.
 5. The eye-controlled apparatusaccording to claim 2, wherein the image processing module and thecalculation module are integrated in the device to be operated.
 6. Theeye-controlled apparatus according to claim 1, wherein the fixationpoint acquisition unit comprises: spectacles; a scene acquisition moduleprovided on the spectacles and adapted to acquire a scene image seen bythe human eyes through the spectacles, the scene image containing animage of the device to be operated; a second human eye image acquisitionmodule provided on the spectacles and adapted to acquire an image of thehuman eyes, a fixation direction determination module configured todetermine a fixation direction of the human eyes based on the image ofthe human eyes acquired by the second human eye image acquisitionmodule; and a fixation point determination module configured todetermine the position information about the fixation point of the humaneyes on the device to be operated based on the scene image acquired bythe scene acquisition module and the fixation direction of the humaneyes determined by the fixation direction determination module.
 7. Theeye-controlled apparatus according to claim 6, wherein the fixationdirection determination module is configured to determine the fixationdirection of the human eyes based on pupil positions in the image of thehuman eyes acquired by the second human eye image acquisition module. 8.The eye-controlled apparatus according to claim 1, wherein the presetaction comprises keeping the position of the fixation point of the humaneyes on the device to be operated unchanged for 2 to 3 seconds, or,blinking rapidly for 3 to 5 times within a preset time while keeping theposition of the fixation point of the human eyes on the device to beoperated unchanged.
 9. The eye-controlled apparatus according to claim1, wherein the human eye action detection unit is an apparatus having animaging function.
 10. The eye-controlled apparatus according to claim 2,wherein the human eye action detection unit is configured to acquire areal-time image of the human eyes by the first human eye imageacquisition module or the second human eye image acquisition module, andobtain the action of the human eyes based on the real-time image of thehuman eyes.
 11. An eye-controlled system comprising: a plurality ofdevices to be operated and an eye-controlled apparatus according toclaim
 1. 12. The eye-controlled system according to claim 11, whereinthe fixation point acquisition unit comprises: an infrared emissionmodule provided on the device to be operated and adapted to emitinfrared light to the human eyes and form light reflection points in twopupils of the human eyes; a first human eye image acquisition moduleprovided on the device to be operated and adapted to acquire an image ofthe human eyes; an image processing module configured to establish animage coordinate system based on the image of the human eyes acquired bythe first human eye image acquisition module, and process the image ofthe human eyes to obtain position coordinates of centers of the twopupils and of the light reflection points in the image coordinatesystem; and a calculation module configured to obtain the positioninformation about the fixation point of the human eyes on the device tobe operated by a cross ratio invariance algorithm based on the positioncoordinates of centers of the two pupils and of the light reflectionpoints in the image coordinate system obtained by the image processingmodule, wherein the fixation point acquisition units in theeye-controlled apparatus have the same number as that of the devices tobe operated, and are one-to-one corresponding to the devices to beoperated; and wherein each fixation point acquisition units is providedon a respective device to be operated.
 13. The eye-controlled systemaccording to claim 12, wherein the human eye action detection units inthe eye-controlled apparatus have the same number as that of the devicesto be operated, and are one-to-one corresponding to the devices to beoperated; and each human eye action detection units is provided on arespective device to be operated.
 14. The eye-controlled systemaccording to claim 12, wherein all of the fixation point acquisitionunits are connected to one single control signal generation unit, thecontrol signal generation unit being adapted to store a plurality ofposition control correspondence tables corresponding to the devices tobe operated, respectively, and transmit control information to eachdevice to be operated.
 15. The eye-controlled system according to claim11, wherein the fixation point acquisition unit comprises: spectacles; ascene acquisition module provided on the spectacles and adapted toacquire a scene image seen by the human eyes through the spectacles, thescene image containing an image of the device to be operated; a secondhuman eye image acquisition module provided on the spectacles andadapted to acquire an image of the human eyes, a fixation directiondetermination module configured to determine a fixation direction of thehuman eyes based on the image of the human eyes acquired by the secondhuman eye image acquisition module; and a fixation point determinationmodule configured to determine the position information about thefixation point of the human eyes on the device to be operated based onthe scene image acquired by the scene acquisition module and thefixation direction of the human eyes determined by the fixationdirection determination module, wherein the eye-controlled apparatuscomprises one fixation point acquisition unit, and wherein theeye-controlled system further comprises a plurality of activation unitsone-to-one corresponding to the devices to be operated, each activationunits being provided on a respective device to be operated, andconfigured to activate the eye-controlled apparatus when the fixationpoint acquisition unit is directed to a device to be operatedcorresponding to the activation unit and control the control signalgeneration unit to call the position control correspondence tablecorresponding to the device to be operated corresponding to theactivation unit.
 16. The eye-controlled system according to claim 15,wherein the eye-controlled apparatus comprises one human eye actiondetection unit provided on spectacles of the fixation point acquisitionunit.
 17. An eye-controlled method, comprising steps of: by a fixationpoint acquisition unit, acquiring the position information about thefixation point of the human eyes on the device to be operated; by ahuman eye action detection unit, detecting whether the human eyes takethe preset action, and controlling the fixation point acquisition unitto transmit the current position information about the fixation point ofthe human eyes on the device to be operated to a control signalgeneration unit when detecting the preset action taken by the humaneyes; and by the control signal generation unit, generating the controlsignal corresponding to the current position information about thefixation point of the human eyes on the device to be operated based onthe pre-stored position control correspondence table corresponding tothe device to be operated, and transmitting the generated control signalto the device to be operated so as to control the device to execute acorresponding operation, wherein the position information on the deviceto be operated and the control signals corresponding to the positioninformation, respectively, are stored in the position controlcorrespondence table.
 18. The eye-controlled method according to claim17, further comprising a step of: prior to acquiring the positioninformation about the fixation point of the human eyes on the device tobe operated by the fixation point acquisition unit, establishing theposition control correspondence table corresponding to the device to beoperated.
 19. The eye-controlled apparatus according to claim 6, whereinthe human eye action detection unit is configured to acquire a real-timeimage of the human eyes by the first human eye image acquisition moduleor the second human eye image acquisition module, and obtain the actionof the human eyes based on the real-time image of the human eyes.